Reconfiguring an acceleration component among interconnected acceleration components

ABSTRACT

The present invention extends to methods, systems, and computer program products for reconfiguring an acceleration component among interconnected acceleration components. Aspects of the invention facilitate reconfiguring an acceleration component among interconnected acceleration components using a higher-level software service. A manager or controller isolates an acceleration component by sending a message to one or more neighbor acceleration components instructing the one or more neighbor acceleration components to stop accepting communication from the acceleration component. The manager or controller can then shut down an application layer at the acceleration component for at least partial reconfiguration and closes input/output (I/O) portions. After reconfiguration completes, communication between the acceleration component and the one or more neighbor acceleration components can resume

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional PatentApplication Ser. No. 62/149,308, entitled “Reconfiguring AccelerationComponents Of A Composed Service”, filed Apr. 17, 2015 which isincorporated herein in its entirety.

BACKGROUND

Computer systems and related technology affect many aspects of society.Indeed, the computer system's ability to process information hastransformed the way we live and work. Computer systems now commonlyperform a host of tasks (e.g., word processing, scheduling, accounting,etc.) that prior to the advent of the computer system were performedmanually. More recently, computer systems have been coupled to oneanother and to other electronic devices to form both wired and wirelesscomputer networks over which the computer systems and other electronicdevices can transfer electronic data. Accordingly, the performance ofmany computing tasks is distributed across a number of differentcomputer systems and/or a number of different computing environments.For example, distributed applications can have components at a number ofdifferent computer systems.

BRIEF SUMMARY

Examples extend to methods, systems, and computer program products forreconfiguring an acceleration component among interconnectedacceleration components. It is determined that it is appropriate toreconfigure a role at the acceleration component based on a detectedcondition at one or more of: the acceleration component and the role.The acceleration component is included in a group of interoperatingacceleration components in a hardware acceleration plane. Roles at eachacceleration component in the group of interoperating accelerationcomponents are linked together to compose a graph that provides serviceacceleration for a service.

The acceleration component is isolated from neighbor accelerationcomponents in the group of interoperating acceleration components. Theacceleration component is isolated by sending a halt command to one ormore neighbor acceleration components of the acceleration component. Thehalt command instructs the one or more neighbor acceleration componentsto ignore communication from the acceleration component. An indicationis received from each of the one or more neighbor accelerationcomponents that the neighbor acceleration component is halted.

The role is reconfigured at the acceleration component. Afterreconfiguration, an activate command is sent to the one or more neighboracceleration components of the acceleration component. The activatecommand instructs the one or more neighbor acceleration components toaccept communication from the acceleration component. An indication isreceived from each of the one or more neighbor acceleration componentsindicating that the neighbor acceleration component is activated.

This summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used as an aid in determining the scope of the claimed subjectmatter.

Additional features and advantages of will be set forth in thedescription which follows, and in part will be obvious from thedescription, or may be learned by practice. The features and advantagesmay be realized and obtained by means of the instruments andcombinations particularly pointed out in the appended claims. These andother features and advantages will become more fully apparent from thefollowing description and appended claims, or may be learned by practiceas set forth hereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to describe the manner in which the above-recited and otheradvantages and features can be obtained, a more particular descriptionwill be rendered by reference to specific implementations thereof whichare illustrated in the appended drawings. Understanding that thesedrawings depict only some implementations and are not therefore to beconsidered to be limiting of its scope, implementations will bedescribed and explained with additional specificity and detail throughthe use of the accompanying drawings in which:

FIG. 1 illustrates an example architecture that includes a softwareplane and a hardware acceleration plane.

FIG. 2 illustrates an example architecture, including servers, that canbe used in a data center

FIG. 3 illustrates an example server.

FIG. 4 illustrates an example server.

FIG. 5 illustrates an example service implemented using components of asoftware plane and components of a hardware acceleration plane.

FIG. 6 illustrates an example service implemented using components of asoftware plane and components of a hardware acceleration plane.

FIG. 7 illustrates an example architecture of an acceleration component.

FIG. 8 illustrates an acceleration component including separateconfigurable domains.

FIG. 9 illustrates functionality for performing data transfer between alocal host component and an associated local hardware accelerationcomponent.

FIG. 10 illustrates an example architecture of a host component.

FIG. 11 illustrates an example architecture of management functionalityfor managing services in a data center.

FIG. 12 illustrates an example architecture with additional componentsof the management functionality of FIG. 11.

FIGS. 13A-13F illustrates an example architecture that facilitatesreconfiguring an acceleration component among interconnectedacceleration components.

FIG. 14 illustrates a flow chart of an example method for reconfiguringan acceleration component among interconnected acceleration components.

DETAILED DESCRIPTION

Examples extend to methods, systems, and computer program products forreconfiguring an acceleration component among interconnectedacceleration components. It is determined that it is appropriate toreconfigure a role at the acceleration component based on a detectedcondition at one or more of: the acceleration component and the role.The acceleration component is included in a group of interoperatingacceleration components in a hardware acceleration plane. Roles at eachacceleration component in the group of interoperating accelerationcomponents are linked together to compose a graph that provides serviceacceleration for a service.

The acceleration component is isolated from neighbor accelerationcomponents in the group of interoperating acceleration components. Theacceleration component is isolated by sending a halt command to one ormore neighbor acceleration components of the acceleration component. Thehalt command instructs the one or more neighbor acceleration componentsto ignore communication from the acceleration component. An indicationis received from each of the one or more neighbor accelerationcomponents that the neighbor acceleration component is halted.

The role is reconfigured at the acceleration component. Afterreconfiguration, an activate command is sent to the one or more neighboracceleration components of the acceleration component. The activatecommand instructs the one or more neighbor acceleration components toaccept communication from the acceleration component. An indication isreceived from each of the one or more neighbor acceleration componentsindicating that the neighbor acceleration component is activated.

Implementations may comprise or utilize a special purpose orgeneral-purpose computer including computer hardware, such as, forexample, one or more processors and system memory, as discussed ingreater detail below. Implementations also include physical and othercomputer-readable media for carrying or storing computer-executableinstructions and/or data structures. Such computer-readable media can beany available media that can be accessed by a general purpose or specialpurpose computer system. Computer-readable media that storecomputer-executable instructions are computer storage media (devices).Computer-readable media that carry computer-executable instructions aretransmission media. Thus, by way of example, and not limitation,implementations can comprise at least two distinctly different kinds ofcomputer-readable media: computer storage media (devices) andtransmission media.

Computer storage media (devices) includes RAM, ROM, EEPROM, CD-ROM,solid state drives (“SSDs”) (e.g., based on RAM), Flash memory,phase-change memory (“PCM”), other types of memory, other optical diskstorage, magnetic disk storage or other magnetic storage devices, or anyother medium which can be used to store desired program code means inthe form of computer-executable instructions or data structures andwhich can be accessed by a general purpose or special purpose computer.

A “network” is defined as one or more data links that enable thetransport of electronic data between computer systems and/or modulesand/or other electronic devices. When information is transferred orprovided over a network or another communications connection (eitherhardwired, wireless, or a combination of hardwired or wireless) to acomputer, the computer properly views the connection as a transmissionmedium. Transmissions media can include a network and/or data linkswhich can be used to carry desired program code means in the form ofcomputer-executable instructions or data structures and which can beaccessed by a general purpose or special purpose computer. Combinationsof the above should also be included within the scope ofcomputer-readable media.

Further, upon reaching various computer system components, program codemeans in the form of computer-executable instructions or data structurescan be transferred automatically from transmission media to computerstorage media (devices) (or vice versa). For example,computer-executable instructions or data structures received over anetwork or data link can be buffered in RAM within a network interfacemodule (e.g., a “NIC”), and then eventually transferred to computersystem RAM and/or to less volatile computer storage media (devices) at acomputer system. Thus, it should be understood that computer storagemedia (devices) can be included in computer system components that also(or even primarily) utilize transmission media.

Computer-executable instructions comprise, for example, instructions anddata which, in response to execution at a processor, cause a generalpurpose computer, special purpose computer, or special purposeprocessing device to perform a certain function or group of functions.The computer executable instructions may be, for example, binaries,intermediate format instructions such as assembly language, or evensource code. Although the subject matter has been described in languagespecific to structural features and/or methodological acts, it is to beunderstood that the subject matter defined in the appended claims is notnecessarily limited to the described features or acts described above.Rather, the described features and acts are disclosed as example formsof implementing the claims.

Those skilled in the art will appreciate that the described aspects maybe practiced in network computing environments with many types ofcomputer system configurations, including, personal computers, desktopcomputers, laptop computers, message processors, hand-held devices,wearable devices, multi-processor systems, microprocessor-based orprogrammable consumer electronics, network PCs, minicomputers, mainframecomputers, mobile telephones, PDAs, tablets, pagers, watches, routers,switches, and the like. The described aspects may also be practiced indistributed system environments where local and remote computer systems,which are linked (either by hardwired data links, wireless data links,or by a combination of hardwired and wireless data links) through anetwork, both perform tasks. In a distributed system environment,program modules may be located in both local and remote memory storagedevices.

The described aspects can also be implemented in cloud computingenvironments. In this description and the following claims, “cloudcomputing” is defined as a model for enabling on-demand network accessto a shared pool of configurable computing resources. For example, cloudcomputing can be employed in the marketplace to offer ubiquitous andconvenient on-demand access to the shared pool of configurable computingresources. The shared pool of configurable computing resources can berapidly provisioned via virtualization and released with low managementeffort or service provider interaction, and then scaled accordingly.

A cloud computing model can be composed of various characteristics suchas, for example, on-demand self-service, broad network access, resourcepooling, rapid elasticity, measured service, and so forth. A cloudcomputing model can also expose various service models, such as, forexample, Software as a Service (“SaaS”), Platform as a Service (“PaaS”),and Infrastructure as a Service (“IaaS”). A cloud computing model canalso be deployed using different deployment models such as privatecloud, community cloud, public cloud, hybrid cloud, and so forth. Inthis description and in the claims, a “cloud computing environment” isan environment in which cloud computing is employed.

In this description and the following claims, an “accelerationcomponent” is defined as a hardware component specialized (configured,possibly through programming) to perform a computing function moreefficiently than software running on general-purpose central processingunit (CPU) could perform the computing function. Acceleration componentsinclude Field Programmable Gate Arrays (FPGAs), Graphics ProcessingUnits (GPUs), Application Specific Integrated Circuits (ASICs), Erasableand/or Complex programmable logic devices (PLDs), Programmable ArrayLogic (PAL) devices, Generic Array Logic (GAL) devices, and massivelyparallel processor array (MPPA) devices.

In this description and in the following claims, a “role” is defined asfunctionality provided by an acceleration component to a group ofinteroperating acceleration components used to accelerate a service.Roles at each acceleration component in a group of interoperatingacceleration components can be linked together to compose a graph thatprovides the service acceleration.

In this description an in the following claims, an “image” is defined asa file including information that can be used in configuration of anacceleration component, such as, for example, an FPGA. Informationincluded in an image file can be used to program hardware components ofan acceleration component (e.g., logic blocks and reconfigurableinterconnects of an FPGA) to implement desired functionality. Desiredfunctionality can be implemented to solve virtually any problem which iscomputable.

In this description and in the following claims, a “graph” is defined asa group of interconnected (e.g., network connected) accelerationcomponents providing acceleration for a service wherein eachacceleration component in the group provides a portion of theacceleration functionality.

In this description and in the following claims, a “neighboracceleration component” is defined as an acceleration componentconfigured to exchange input and/or output with another accelerationcomponent when interconnected to the other acceleration component withina graph. Neighbor is viewed logically from the perspective of the graph.The physical proximity of an acceleration component relative to anotheracceleration component is not a determining factor in identifyingneighbor acceleration components. That is, acceleration components thatare not physically adjacent to one another (or even near to one anotheron a network or within a datacenter) can be configured to exchange datawith one another when interconnected within a graph. Accelerationcomponents interconnected within a graph can be viewed as neighboracceleration components even if data exchanged between the accelerationcomponents physically passes through other acceleration componentsoutside of the graph or through host components in transit between theacceleration components. However, acceleration components that arephysically adjacent or near to one another on a network or in adatacenter and are interconnected within a graph can also be viewed asneighbor acceleration components with respect to one another.

In general, an acceleration component can include an array ofprogrammable logic blocks and hierarchy of reconfigurable interconnectsthat allow logic blocks to be connected together in differentconfigurations to provide different functionality (i.e., differentroles). Image files can be received and loaded at an accelerationcomponent to configure programmable logic blocks and configureinterconnects to provide desired functionality (i.e., roles).

In some environments, applications (services) are provided to a clientfrom a data center. A data center includes multiple (and potentially avery large number of) software-driven general purpose computing devices.Each general purpose computing device can include one or more centralprocessing units (CPUs) that process machine-readable instructions toperform specified computing activities. The multiple general purposecomputing devices are networked to one another such that the combinedpower of the multiple general purpose computer systems (or subsetsthereof) can be used to perform more complex computing activities.

Data center providers face continuing challenges to increase processingcapabilities and efficiency within and provided by data centers.Continuing to add more and more general purpose computing devices is notfeasible due at least in part to power limitations. Computing devicespecialization is one option. For example, computing devices can bespecialized for specific scale workloads to provide some efficiencygains. However, computing device specialization is problematic for atleast two reasons. First, lack of homogeneity in a data center increasesmanagement issues and provides inconsistent platforms for applicationsto rely on. Further, data center services evolve rapidly, makingnon-programmable hardware features impractical. Thus, data centerproviders need continued improvements in performance and efficiency butcannot obtain those improvements from general purpose computing devices.

Aspects facilitate reconfiguring a single acceleration component amonginterconnected acceleration components using a higher-level softwareservice. A manager or controller can send a message to an accelerationcomponent to instruct the acceleration component not to listen toneighbor acceleration components until an image is loaded onto theacceleration component. The manager or controller then isolates theacceleration component by sending a message to the neighbor accelerationcomponents instructing the neighbor acceleration components to notlisten to the acceleration component. The manager controller can thenshut down an application layer at the acceleration component for atleast partial reconfiguration and closes input/output (I/O) portions.When reconfiguration is complete, communication between the accelerationcomponent and neighbor acceleration components is resumed.

Accordingly, aspects can be used to: (a) reconfigure a singleacceleration component among interconnected acceleration components, (b)reconfigure an acceleration component using higher-level softwareservices, (c) send a message to an acceleration component instructingthe acceleration component to ignore neighbor acceleration componentsuntil the acceleration component is reconfigured, (d) isolate anacceleration component by sending a message to neighboring accelerationcomponents instructing the neighboring acceleration components to ignorethe acceleration component, (e) shut down the application layer of anacceleration component for at least partial configuration, and (f)reconfigure the application layer of an acceleration component and closeI/O portions of an acceleration component.

Aspects can use a performance handling algorithm that can reconfiguregroups of acceleration components or remap services robustly, recoverfrom degraded performance and failures by remapping accelerationcomponents, and report errors to management software to diagnoseproblems.

In general, a data center deployment includes a hardware accelerationplane and a software plane. The hardware acceleration plane can includea plurality of networked acceleration components (e.g., FPGAs). Thesoftware plane can include a plurality of networked software-implementedhost components (e.g., central processing units (CPUs)). A networkinfrastructure can be shared between the hardware acceleration plane andthe software plane. In some environments, software-implemented hostcomponents are locally linked to corresponding acceleration components.

FIG. 1 illustrates an example architecture 102 that includes a softwareplane 104 and a hardware acceleration plane 106. The software plane 104includes a collection of software-driven components (each denoted by thesymbol “S”) while the hardware plane includes a collection of hardwareacceleration components (each denoted by the symbol “H”). For example,each host component may correspond to a server computer that executesmachine-readable instructions using one or more central processing units(CPUs). Each CPU, in turn, may execute the instructions on one or morehardware threads. Each acceleration component can execute hardware logicfor implementing functions, such as, for example, portions of servicesoffer by a data center.

Hardware acceleration plane 106 can be constructed using a heterogeneouscollection of acceleration components, including different types ofacceleration components and/or the same type of acceleration componentswith different capabilities. For example, hardware acceleration plane106 can include FPGA devices having different respective processingcapabilities and architectures, a mixture of FPGA devices and otherdevices, and so on. Hardware acceleration plane 106 provides areconfigurable fabric of acceleration components.

A host component generally performs operations using a temporalexecution paradigm (i.e., sequentially) by using each of its CPUhardware threads to execute machine-readable instructions, one after theafter. In contrast, an acceleration component may perform operationsusing a spatial paradigm (i.e., concurrently) by using a large number ofparallel logic elements to perform computational tasks. Thus, anacceleration component can perform some operations in less time comparedto a software-driven host component. In the context of the architecture102, the “acceleration” qualifier associated with the term “accelerationcomponent” reflects its potential for accelerating the functions thatare performed by the host components.

In one example, architecture 102 corresponds to a data centerenvironment that includes a plurality of computer servers. The computerservers correspond to the host components in the software plane 104. Inanother example, architecture 102 corresponds to an enterprise system.In a further example, the architecture 102 corresponds to a user deviceor appliance which uses at least one host component that has access totwo or more acceleration components, etc. Other implementations forarchitecture 102 are also possible.

Common network infrastructure 120 couples host components in thesoftware plane 104 to other host components and couples accelerationcomponents in the hardware acceleration plane 106 to other accelerationcomponents. That is, host components can use common networkinfrastructure 120 to interact with one another and accelerationcomponents can use common network infrastructure 120 to interact withone another. Interaction among host components in the software plane 104is independent of the interaction among acceleration components in thehardware acceleration plane 106. As such, two or more accelerationcomponents may communicate in a transparent manner relative to hostcomponents in the software plane 104, outside the direction of the hostcomponents, and without the host components being “aware” of particularinteraction is even taking place in the hardware acceleration plane 106.

Architecture 102 can use any of a variety of different protocols tofacilitate communication between acceleration components over networkinfrastructure 120 and can use any of a variety of different protocolsto facilitate communication between host components over networkinfrastructure 120. For example, architecture 102 can uses Ethernetprotocol to transmit Internet Protocol (IP) packets over networkinfrastructure 120. In one implementation, each local host component ina server is given a single physical IP address. The local accelerationcomponent in the same server may adopt the same IP address. The servercan determine whether an incoming packet is destined for the local hostcomponent or destined for the local acceleration component in differentways. For example, packets that are destined for the local accelerationcomponent can be formulated as UDP packets having a specific port;host-defined packets, on the other hand, may not be formulated in thisway. In another example, packets belonging to the acceleration plane 106can be distinguished from packets belonging to the software plane 104based on the value of a status flag in each of the packets.

As such, architecture 102 can be viewed as two logical networks(software plane 104 and hardware acceleration plane 106) that share thesame physical network communication links. Packets associated with thetwo logical networks may be distinguished from each other by theirrespective traffic classes.

In another aspect, each host component in the architecture 102 iscoupled to at least one acceleration component in hardware accelerationplane 104 through a local link. For example, a host component andacceleration component can be arranged together and maintained as singleserviceable unit (e.g., a server) within architecture 102. In thisarrangement, the server can be referred to as the “local” host componentto distinguish it from other host components that are associated withother servers. Similarly, acceleration component(s) of a server can bereferred to as the “local” acceleration component(s) to distinguish themfrom other acceleration components that are associated with otherservers.

As depicted in architecture 102, host component 108 is coupled toacceleration component 110 through a local link 112 (e.g., a PeripheralComponent Interconnect Express (PCIe) link). Thus, host component 108 isa local host component form the perspective of acceleration component110 and acceleration component 110 is a local acceleration componentfrom the perspective of host component 108. The local linking of hostcomponent 108 and acceleration component 110 can form part of a server.More generally, host components in software plane 104 can be locallycoupled to acceleration components in hardware acceleration plane 106through many individual links collectively represented as alocal_(H)-to-local_(S) coupling 114.

Thus, a host component can interact directly with any locally linkedacceleration components. As such, a host component can initiatecommunication to a locally linked acceleration component to causefurther communication among multiple acceleration components. Forexample, a host component can issue a request for a service (or portionthereof) where functionality for the service (or portion thereof) iscomposed across a group of one or more acceleration components inhardware acceleration plane 106.

Thus, a host component can also interact indirectly with otheracceleration components in hardware acceleration plane 106 to which thehost component is not locally linked. For example, host component 108can indirectly communicate with acceleration component 116 viaacceleration component 110. More specifically, acceleration component110 communicates with acceleration component 116 via a link 118 (e.g.,network infrastructure 120).

Acceleration components in hardware acceleration plane 106 can be usedto accelerate larger-scale services robustly in a data center.Substantial portions of complex datacenter services can be mapped toacceleration components (e.g., FPGAs) by using low latency interconnectsfor computations spanning multiple acceleration components. Accelerationcomponents can also be reconfigured as appropriate to provide differentservice functionality at different times.

FIG. 2 illustrates an example architecture 202 that can be used in adata center. Servers 204, 206, and 208 can be included in a rack in thedata center. Each of servers 204, 206, and 208 can be coupled totop-of-rack (TOR) switch 210. Other racks, although not shown, may havea similar configuration. Server 204 further includes host component 212including CPUs 214, 216, etc. Host component 212 along with hostcomponents from servers 206 and 208 can be included in software plane104. Server 204 also includes acceleration component 218. Accelerationcomponent 218 along with acceleration components from servers 206 and208 can be included in hardware acceleration plane 106.

Acceleration component 218 is directly coupled to host component 212 vialocal link 220 (e.g., a PCIe link). Thus, host component 212 can viewacceleration component 218 as a local acceleration component andacceleration component 218 can view host component 212 as a local hostcomponent. Acceleration component 218 and host component 212 are alsoindirectly coupled by way of network interface controller 222 (e.g.,used to communicate across network infrastructure 120). Server 204 canload images representing service functionality onto accelerationcomponent 218.

Acceleration component 218 is also coupled to TOR switch 210. Hence, inarchitecture 202, acceleration component 218 represents the path throughwhich host component 212 interacts with other components in the datacenter (including other host components and other accelerationcomponents). Architecture 202 allows acceleration component 218 toperform processing on packets that are received from (and/or sent to)TOR switch 210 (e.g., by performing encryption, compression, etc.),without burdening the CPU-based operations performed by host component212.

Management functionality 232 serves to manage the operations ofarchitecture 202. Management functionality 232 can be physicallyimplemented using different control architectures. For example, in onecontrol architecture, the management functionality 232 may includeplural local management components that are coupled to one or moreglobal management components.

FIG. 3 illustrates an example server 302. Server 302 includes hostcomponent 304 including CPUs 306, 308, etc., acceleration component 310,and local link 312. Acceleration component 310 is directly coupled tohost component 304 via local link 312 (e.g., a PCIe link). Thus, hostcomponent 304 can view acceleration component 310 as a localacceleration component and acceleration component 310 can view hostcomponent 304 as a local host component. Host component 304 andacceleration component 310 can be included in software plane 104 andhardware acceleration plane 106 respectively. Server 302 implementsnetwork interface controller (NIC) 314 as an internal component ofacceleration component 310. Server 302 can load images representingservice functionality onto acceleration component 310.

FIG. 4 illustrates an example server 402. Server 402 includes hostcomponents 404 through 406 including any number n of host components.Host components 404 through 406 can be included in software plane 104.Server 402 includes acceleration components 408 through 410 includingany number m of acceleration components. Acceleration components 408through 410 can be included in hardware acceleration plane 106. Server402 can also include a network interface controller (not shown).

Server 402 can include a single host component locally linked to twoacceleration components. The two acceleration components can performdifferent respective tasks. For example, one acceleration component canbe used to process outgoing traffic to its local TOR switch, while theother acceleration component can be used to process incoming trafficfrom the TOR switch. In addition, server 402 can load imagesrepresenting service functionality onto any of the accelerationcomponents 408 through 410.

In general, a service (e.g., document ranking, encryption, compression,computer vision, speech translation, computer vision, machine learning,etc.) can be implemented at one or more host components, at one or moreacceleration components, or a combination of one or more host componentsand one or more acceleration components depending on what components arebetter suited to provide different portions of the service.

FIG. 5 illustrates an example service 512 implemented using componentsof software plane 104 and components of hardware acceleration plane 106.In operation (1), host component 502 communicates with host component504 in the course of performing a computational task. In operation (2)host component 504 then requests the use of service 512 that isimplemented in the hardware acceleration plane 106 (although hostcomponent 504 may not be “aware” of where service 512 is implemented) bycommunicating with acceleration component 506 over a local link.

The requested service 512 is a composed service spread out over aplurality of acceleration components, each of which performs a specifiedportion of the service. Although acceleration component 506 wascontacted to request use of the service 512, acceleration component 506may not be the head of the composed service (or even be part of themulti-component service). Instead, acceleration component 508 may be thehead component for the composed service.

As such, in operation (3), host component 504 indirectly communicateswith acceleration component 508 via acceleration component 506.Acceleration component 508 then performs its portion of the composedservice to generate an intermediate output result. In operation (4),acceleration component 508 then invokes acceleration component 510,which performs another respective portion of the composed service, togenerate a final result. In operations (5), (6), and (7), the hardwareacceleration plane 106 successively forwards the final result back tothe requesting host component 504, through the same chain of componentsset forth above but in the opposite direction.

Operations in hardware acceleration plane 106 are performed in anindependent manner of operations performed in the software plane 104. Inother words, the host components in the software plane 104 do not managethe operations in the hardware acceleration plane 106. However, the hostcomponents may invoke the operations in the hardware acceleration plane106 by issuing requests for services that are hosted by the hardwareacceleration plane 106.

The hardware acceleration plane 106 operates in a manner that istransparent to a requesting host component. For example, host component504 may be “unaware” of how its request is being processed in hardwareacceleration plane 106, including the fact that the service correspondsto a composed service.

Communication in software plane 104 (e.g., corresponding to operation(1)) can take place using the same common network infrastructure 120 ascommunication in the hardware acceleration plane 106 (e.g., correspondto operations (3)-(6)). Operations (2) and (7) may take place over alocal link, corresponding to the local_(H)-to-local_(S) coupling 114shown in FIG. 1.

FIG. 6 illustrates an example service 612 implemented using componentsof a software plane and components of hardware acceleration plane 106Service 612 uses a different flow structure than service 512. Morespecifically, in operation (1), a host component (not shown) sends arequest to its local acceleration component 602. In this example, localacceleration component 602 is also the head component of service 612. Inoperation (2), local acceleration component 602 may then forward one ormore messages to a plurality of respective acceleration components. Eachacceleration component that receives a message may perform a portion ofa composed service in parallel with the other acceleration components.(FIG. 6 may represent only a portion of service 612, other portions ofservice 612 can be implemented at other hardware accelerators).

In general, an acceleration component can include any of variety ofcomponents some of which can be more or less common across differentapplication images. Some components, such as, for example, a role, aredistinct between application images. Other components, such as, forexample, routers, transport components, switches, diagnostic recorders,etc., can be relatively common between some number of applicationimages. These other relatively common components can be viewed as beingincluded in an intermediate layer of abstraction or “soft shell”.Further components, such as, for example, bridges, bypass controls,Network Interface Cards, Top of Rack Interfaces, buffers, memorycontrollers, PCIe controllers, Inter-FPGA network controllers,configuration memories and interfaces, host interfaces, debugging andback-channel interfaces (e.g., Joint Test Action Group (JTAG)interfaces, Inter-Integrated Circuit (I2C) interfaces, etc.), sensors,etc. can be very common between a higher number of (and essentially all)application images. These further very common components can be viewedas included in a greater layer of abstraction (e.g., than the otherrelatively common components) or “shell”.

When an acceleration component (e.g., an FPGA) is reconfigured with newfunctionality, it is likely (although not guaranteed) that a role (i.e.,the application-specific logic) at the acceleration component ischanged. However, it is unlikely that existing functionality in the softshell is changed and it is extremely unlikely that existingfunctionality in the soft shell is changed. Thus, components in the softshell and to greater extent components in the shell provide a commoninterface for a role. As such, the shell allows code for a role to beported relatively easy across different acceleration components.

Turning to FIG. 7, FIG. 7 illustrates an example architecture of anacceleration component 702. Acceleration component 702 can be includedin hardware acceleration plane 106. Components included in accelerationcomponent 702 can be implemented on hardware resources (e.g., logicblocks and programmable interconnects) of acceleration component 702.

Acceleration component 702 includes application logic 706, soft shell704 associated with a first set of resources and shell 711 associatedwith a second set of resources. The resources associated with shell 711correspond to lower-level interface-related components that generallyremain the same across many different application scenarios. Theresources associated with soft shell 704 can remain the same across atleast some different application scenarios. The application logic 706may be further conceptualized as including an application domain (e.g.,a “role”). The application domain or role can represent a portion offunctionality included in a composed service spread out over a pluralityof acceleration components.

The application domain hosts application logic 706 that performs servicespecific tasks (such as a portion of functionality for rankingdocuments, encrypting data, compressing data, facilitating computervision, facilitating speech translation, machine learning, etc.).Resources associated with soft shell 704 are generally less subject tochange compared to the application resources, and the resourcesassociated with shell 711 are less subject to change compared to theresources associated with soft shell 704 (although it is possible tochange (reconfigure) any component of acceleration component 702).

In operation, application logic 706 interacts with the shell resourcesand soft shell resources in a manner analogous to the way asoftware-implemented application interacts with its underlying operatingsystem resources. From an application development standpoint, the use ofcommon shell resources and soft shell resources frees a developer fromhaving to recreate these common components for each service.

Referring first to the shell 711, shell resources include bridge 708 forcoupling acceleration component 702 to the network interface controller(via a NIC interface 710) and a local top-of-rack switch (via a TORinterface 712). Bridge 708 also includes a data path that allows trafficfrom the NIC or TOR to flow into acceleration component 702, and trafficfrom the acceleration component 702 to flow out to the NIC or TOR.Internally, bridge 708 may be composed of various FIFOs (714, 716) whichbuffer received packets, and various selectors and arbitration logicwhich route packets to their desired destinations. A bypass controlcomponent 718, when activated, can control bridge 708 so that packetsare transmitted between the NIC and TOR without further processing bythe acceleration component 702.

Memory controller 720 governs interaction between the accelerationcomponent 702 and local memory 722 (such as DRAM memory). The memorycontroller 720 may perform error correction as part of its services.

Host interface 724 provides functionality that enables accelerationcomponent 702 to interact with a local host component (not shown). Inone implementation, the host interface 724 may use Peripheral ComponentInterconnect Express (PCIe), in conjunction with direct memory access(DMA), to exchange information with the local host component. The outershell may also include various other features 726, such as clock signalgenerators, status LEDs, error correction functionality, and so on.

Turning to soft shell 704, router 728 is for routing messages betweenvarious internal components of the acceleration component 702, andbetween the acceleration component and external entities (e.g., via atransport component 730). Each such endpoint is associated with arespective port. For example, router 728 is coupled to memory controller720, host interface 724, application logic 706, and transport component730.

Transport component 730 formulates packets for transmission to remoteentities (such as other acceleration components), and receives packetsfrom the remote entities (such as other acceleration components). A3-port switch 732, when activated, takes over the function of the bridge708 by routing packets between the NIC and TOR, and between the NIC orTOR and a local port associated with the acceleration component 702.

Diagnostic recorder 734 can store information regarding operationsperformed by the router 728, transport component 730, and 3-port switch732 in a circular buffer. For example, the information may include dataabout a packet's origin and destination IP addresses, host-specificdata, timestamps, etc. A technician may study a log of the informationin an attempt to diagnose causes of failure or sub-optimal performancein the acceleration component 702.

A plurality of acceleration components similar to acceleration component702 can be included in hardware acceleration plane 106.

Acceleration components can use different network topologies (instead ofusing common network infrastructure 120 for communication) tocommunicate with one another. In one aspect, acceleration components areconnected directly to one another, such as, for example, in a twodimensional torus.

FIG. 8 illustrates an acceleration component 802 including separateconfigurable domains 804, 806, etc. A configuration component (notshown) can configure each configurable domain without affecting otherconfigurable domains. Hence, the configuration component can configureone or more configurable domains while the other configurable domainsare executing operations based on their respective configurations, whichare not disturbed.

FIG. 9 illustrates functionality for performing data transfer between ahost component 902 and an associated (e.g., locally linked) accelerationcomponent 904. Data can be transferred via a host interface (e.g., hostinterface 724), for example, using PCIe in conjunction with DMA memorytransfer). In operation (1), host logic 906 places data to be processedinto kernel-pinned input buffer 908 in main memory associated with thehost logic 906. In operation (2), the host logic 906 instructs theacceleration component 904 to retrieve the data and begin processing it.The host logic's thread is then either put to sleep until it receives anotification event from the acceleration component 904, or it continuesprocessing other data asynchronously. In operation (3), the accelerationcomponent 904 transfers the data from the host logic's memory and placesit in an acceleration component input buffer 910.

In operations (4) and (5), the application logic 912 retrieves the datafrom the input buffer 910, processes it to generate an output result,and places the output result in an output buffer 914. In operation (6),the acceleration component 904 copies the contents of the output buffer914 into output buffer 916 (in the host logic's memory). In operation(7), acceleration component 904 notifies the host logic 906 that thedata is ready for it to retrieve. In operation (8), the host logicthread wakes up and consumes the data in the output buffer 916. Hostlogic 906 may then discard the contents of the output buffer 916, whichallows the acceleration component 904 to reuse it in the next loadingoperation.

FIG. 10 illustrates an example architecture of a host component 1002.Host component 1002 can include one or more processing devices 1004,such as one or more central processing units (CPUs). Host component 1002can also include any storage resources 1006 for storing any kind ofinformation, such as code, settings, data, etc. Without limitation, forinstance, storage resources 1006 may include any of RAM of any type(s),ROM of any type(s), flash devices, hard disks, optical disks, and so on.More generally, any storage resource can use any technology for storinginformation. Further, any storage resource may provide volatile ornon-volatile retention of information. Further, any storage resource mayrepresent a fixed or removable component of host component 1002. In onecase, host component 1002 may perform any of the operations associatedwith local tenant functionality when processing devices 1004 carry outassociated instructions stored in any storage resource or combination ofstorage resources. Host component 1002 also includes one or more drivemechanisms 1008 for interacting with any storage resource, such as ahard disk drive mechanism, an optical disk drive mechanism, and so on.

Host component 1002 also includes an input/output module 1010 forreceiving various inputs (via input devices 1012), and for providingvarious outputs (via output devices 1014). One particular outputmechanism may include a presentation device 1016 and an associatedgraphical user interface (GUI) 1018. Host component 1002 can alsoinclude one or more network interfaces 1020 for exchanging data withother devices via one or more communication conduits 1022. One or morecommunication buses 1024 communicatively couple the above-describedcomponents together.

Communication conduit(s) 1022 can be implemented in any manner, e.g., bya local area network, a wide area network (e.g., the Internet),point-to-point connections, etc., or any combination thereof.Communication conduit(s) 1022 can include any combination of hardwiredlinks, wireless links, routers, gateway functionality, name servers,etc., governed by any protocol or combination of protocols.

A plurality of host components similar to host component 1002 can beincluded in software plane 104.

FIG. 11 illustrates an example architecture 1102 of managementfunctionality 1122 for managing services in a data center. Managementfunctionality 1122 can include a number of sub-components that performdifferent respective functions (which can be physically implemented indifferent ways). Architecture 1102 can be included in architecture 102.As such, management functionality 1122 as well as other associatedcomponents can be implemented on hardware resources of a host component(e.g., in software plane 104) and/or implemented on hardware resourcesof an acceleration component (e.g., in hardware acceleration plane 106).Host component hardware resources can include any of the hardwareresources associated with host component 1002. Acceleration componenthardware resources can include any of the hardware resources associatedwith acceleration component 702.

A local determination component 1124, for example, can identify thecurrent locations of services within architecture 102, based oninformation stored in a data store 1126. In operation, locationdetermination component 1124 may receive a request for a service. Inresponse, location determination component 1124 returns an addressassociated with the service, if that address is present in data store1126. The address may identify a particular acceleration component inhardware acceleration plane 106 that hosts (or is the head of) therequested service.

Request handling component (RHC) 1128 processes requests for servicesmade by instances of tenant functionality. For example, an instance oftenant functionality may correspond to a software program running on aparticular local host component. That software program may request aservice in the course of its execution. The RHC 1128 handles the requestby determining an appropriate component in architecture 102 to providethe service. Possible components for consideration include: a localacceleration component (associated with the local host component fromwhich the request originated); a remote acceleration component; and/orthe local host component itself (whereupon the local host componentimplements the service in software). RHC 1128 makes its determinationsbased on one or more request handling considerations, such as whetherthe requested service pertains to a line-rate service. Further, the RHC1128 may interact with the location determination component 1124 inperforming its functions.

A global service allocation component (GSAC) 1130 can operate in abackground and global mode, allocating services to accelerationcomponents based on global conditions in architecture 102 (rather thanhandling individual requests from instances of tenant functionality, asdoes RHC 1128). For example, GSAC 1130 may invoke its allocationfunction in response to a change in demand that affects one or moreservices. GSAC 1130 makes its determinations based on one or moreallocation considerations, such as the historical demand associated withthe services, etc. Further, the GSAC 1130 may interact with the locationdetermination component 1124 in performing its functions. Asub-component of GSAC 1130 can also manage multi-component and/orcomposed services. A multi-component service is a service that iscomposed of plural parts. Plural respective acceleration componentsperform the respective parts.

FIG. 12 illustrates an example architecture with additional componentsof management functionality 1122. As described, location determinationcomponent 1124 identifies the current location of services withinarchitecture 102, based on information stored in the data store 1126. Inoperation, the location determination component 1124 receives a requestfor a service. In response, it returns the address of the service, ifpresent within the data store 1126. The address may identify aparticular acceleration component that implements the service.

Request handling component (RHC) 1128 handles requests for services bytenant functionality that resides on the host components. In response toeach request by a local host component, RHC 1128 determines anappropriate component to implement the service. For example, RHC 1128may choose from among: a local acceleration component (associated withthe local host component that made the request), a remote accelerationcomponent, or the local host component itself (whereupon the local hostcomponent will implement the service in software), or some combinationthereof. RHC 1128 performs its determinations based on one or morerequest handling considerations.

General allocation component (GSAC) 1130, on the other hand, operates byglobally allocating services to acceleration components withinarchitecture 102 to meet overall anticipated demand in the dataprocessing system and/or to satisfy other objectives (rather thanindividual requests by host components). In performing its functions,the GSAC component 1130 may draw on a data store 1202 that provides freepool information. The free pool information identifies accelerationcomponents that have free capacity to implement one or more services.The GSAC 1130 can also receive input information that has a bearing onits allocation decisions. One such piece of input information pertainsto historical demand information associated with a service, e.g., asmaintained in a data store 1204.

GSAC 1130 and RHC 1128 may use, in part, common logic in reaching theirallocation decisions, and that common logic may, in part, taken intoaccount similar allocation considerations. Further, both RHC 1128 andGSAC 1130 interact with the location determination component 124 in thecourse of performing their respective operations. Otherwise, asdescribed, the GSAC 1130 frames its decisions in a global context,whereas the RHC 1128 is an on-demand component that is primarily focusedon satisfying specific requests.

Configuration component 1206 configures acceleration components, forexample, by sending a configuration stream to the accelerationcomponents. A configuration stream specifies the logic (e.g., an image)to be “programmed” into a recipient acceleration component. Theconfiguration component 1206 may use different strategies to configurean acceleration component.

The failure monitoring component 1208 determines whether a previouslyconfigured acceleration component has failed. Various components of themanagement functionality 1122 may respond to failure notification bysubstituting a spare acceleration component for a failed accelerationcomponent, reconfiguring an acceleration component, partialreconfiguring acceleration component, reloading data in an accelerationcomponent, etc.

As described, service acceleration can be provided by linking roles froma group of interoperating acceleration components. The linked roles canbe composed in a directed graph in any variety of different ways,including a directed acyclic graph, a directed cyclic graph, etc., toprovide service functionality and/or acceleration. For example, in someaspects, linked roles are composed in a pipeline or ring.

FIGS. 13A-13F illustrates an example architecture 1300 that facilitatesreconfiguring an acceleration component among interconnectedacceleration components. Referring initially to FIG. 13A, computerarchitecture 1300 includes acceleration components 1301-1304, othercomponents 1331 and 1332, and service manager 1322. Accelerationcomponents 1301-1304 (e.g., FPGAs), other components 1331 and 1332, andservice manager 1322 can be connected to (or be part of) a network, suchas, for example, a Local Area Network (“LAN”), a Wide Area Network(“WAN”), and even the Internet. Accordingly, acceleration components1301-1304, other components 1331 and 1332, and service manager 1322. aswell as any other connected computer systems and their components, cancreate message related data and exchange message related data (e.g.,Internet Protocol (“IP”) datagrams and other higher layer protocols thatutilize IP datagrams, such as, Transmission Control Protocol (“TCP”),Hypertext Transfer Protocol (“HTTP”), Simple Mail Transfer Protocol(“SMTP”), Simple Object Access Protocol (SOAP), etc. or using othernon-datagram protocols) over the network.

In one aspect, acceleration components 1301-1304, other components 1331and 1332, and service manager 1322 are connected to networkinfrastructure 120. Acceleration components 1301-1304 can be included inhardware acceleration plane 106. Other components 1331 and 1332 caninclude host components and/or other acceleration components. The hostcomponents and/or other acceleration components can be included insoftware plane 104 and/or hardware acceleration plane 106 respectively.

Each of acceleration components 1301-1304 can include an array ofprogrammable logic blocks and hierarchy of reconfigurable interconnectsthat allow logic blocks to be connected together in differentconfigurations to provide different functionality (i.e., differentroles). Image files can be received and loaded at acceleration componentacceleration components 1301-1304 to configure programmable logic blocksand configure interconnects to provide desired functionality (i.e.,roles).

As depicted, acceleration components 1301, 1302, 1303, and 1304 areprogrammed with roles 1311, 1312, 1313, and 1314 respectively. Roles1311, 1312, 1313, and 1314 are linked together to compose graph 1333.Graph 1333 provides service acceleration for a service, such as, forexample, document ranking, encryption, compression, computer vision,speech translation, machine learning, etc. The depicted arrangement ofacceleration components 1301, 1302, 1303, and 1304 is logical. Thephysical proximity of acceleration components 1301, 1302, 1303, and 1304relative to one another can vary (e.g., same server, different serverssame rack, different racks, etc.).

Input and output from roles at (logically) neighboring accelerationcomponents may depend on one another or input and output from othercomponents 1331 and/or other components 1332 (e.g., host components orfunctionality composed from a different group of interoperatingacceleration components). For example, input to role 1313 can depend onoutput from role 1312. Similarly, input to role 1314 can depend onoutput from role 1313.

Some or all of acceleration components 1301, 1302, 1303, and 1304 canparticipate in one-way or two-communication with (logically) neighboringacceleration components and other components (e.g., host components).Thus, input and output from one acceleration component can depend oninput and output from another acceleration component and vice versa. Forexample, input to role 1312 can depend on output from role 1313 andinput to role 1311 can depend on output from role 1312.

Graph 1333 can provide service acceleration for an entire service orpart of a service. For example, graph 1333 can be used to accelerationpart of a document ranking service used to provide search engineresults. Graph 1333 can interoperate with other portions of servicefunctionality composed from other groups of interoperating accelerationcomponents and/or provided by one or more host components. For example,for a document ranking service, acceleration for document featureextraction can be composed from one group of interoperating accelerationcomponents, acceleration for free form expression calculations can becomposed from another group of interoperating acceleration components,and acceleration for scoring calculations can be composed from a furthergroup of interoperating acceleration components.

In addition to the expressly depicted roles 1311, 1312, 1313, and 1314,one or more of acceleration components 1301, 1302, 1303, and 1304 canalso be programmed with one or more other roles. The one or more otherroles can be configured to provide part of other service functionality,for example, relating to accelerating encryption, compression, computervision, speech translation, machine learning, etc. Thus, one or more ofacceleration components 1301, 1302, 1303, and 1304 may simultaneously becomposed into one or more different groups of interoperatingacceleration components that provide different functionality fordifferent services.

In general, service manager 1322 is configured to monitor networkinfrastructure 120. When appropriate, service manager 1322 canreconfigure acceleration components in hardware acceleration plane 106to restore service acceleration for a service. From time to time, forexample, due to a detected error or failure, for installation of anupgrade, etc., a role at an acceleration component can be reconfigured.Service manager 1322 can manage a reconfiguration protocol for orderlyreconfiguration of a role. Service manager 1322 can include any of thecomponents and functionality described with respect to managementfunctionality 232 and management functionality 1122 as well as any otherappropriate components and functionality for implementing areconfiguration protocol.

FIG. 14 illustrates a flow chart of an example method 1400 forreconfiguring an acceleration component among interconnectedacceleration components. Method 1400 will be described with respect tothe components and data of architecture 1300.

Method 1400 includes determining that it is appropriate to reconfigure arole at the acceleration component based on a detected condition at oneor more of: the acceleration component and the role, the accelerationcomponent included in a group of interoperating acceleration componentsin a hardware acceleration plane, roles at each acceleration componentin the group of interoperating acceleration components linked togetherto compose a graph that provides service acceleration for a service(1401). For example, turning to FIG. 13B, service manager 1322 candetermine that role 1313 is to be reconfigured based on a detectedcondition at acceleration component 1303 and/or role 1313. A detectedcondition can include an error at acceleration component 1303 and/orrole 1313, need to install an upgrade for role 1313, etc.

Method 1400 includes isolating the acceleration component from neighboracceleration components in the group of interoperating accelerationcomponents, the acceleration component isolated by sending a haltcommand to one or more neighbor acceleration components of theacceleration component, the halt command instructing the one or moreneighbor acceleration components to ignore communication from theacceleration component (1402) and receiving an indication from each ofthe one or more neighbor acceleration components that the neighboracceleration component is halted (1403). For example, service manager1322 can send graph quiesce command 1351 to graph 1333. Graph quiescecommand 1351 instructs graph 1333 to stop all traffic flowing into andout of graph 1333. When graph 1333 is quiesced (i.e., stopped), graph1333 can return graph quiesced message 1361 back to management service1322.

Service manager 1322 can isolate acceleration component 1303 fromacceleration components 1302 and 1304. Service manager 1322 can sendhalt commands 1352 and 1353 to acceleration components 1302 and 1304respectively. Halt commands 1352 and 1353 instruct roles 1312 and 1314to ignore any data coming out of role 1313. Service manager 1322 cansend a similar halt message to acceleration component 1303 to instructacceleration component 1303 to ignore any data coming out of roles 1312and 1314.

Turning to FIG. 13C, acceleration components 1302 and 1304 can returnhalted messages 1362 and 1363 respectively when roles 1312 and 1314 arehalted. Acceleration component 1303 can return a similar halted messagewhen role 1313 is halted. Once acceleration components 1302 and 1304(and 1303) are halted, acceleration component 1303 is essentiallyisolated from other acceleration components of graph 1333.

Method 1400 includes reconfiguring the role at the accelerationcomponent (1404). For example, service manager 1322 can sendreconfiguration command 1354 to acceleration component 1303.Reconfiguration command 1354 causes acceleration component 1303 toreconfigure the functionality of role 1313. Role 1313 can bereconfigured to correct an error or failure, to install an update, etc.

Reconfiguring role 1313 can include loading a new image (either of role1313 or another role) at acceleration component 1303, loadingconfiguration data at acceleration component 1303, remapping role 1313to a different acceleration component, etc. In one aspect, an image isloaded from storage at a locally linked host component. In anotheraspect, an image is loaded from a network location through a networkinterface programmed into acceleration component 1303. Loadingconfiguration data can be used to switch between different portions of asuperset of functionality at acceleration component 1303. The supersetof functionality can be capable of providing a plurality of differentroles.

During reconfiguration of role 1313, acceleration component 1303 mayemit spurious data. For example, pins on an FPGA can flip betweendifferent values. The spurious data may appear as legitimatecommunication (e.g., appropriately formed packets) to role 1312 and/orrole 1314. However, since roles 1312 and 1314 are halted and graph 1333is quiesced, the risk of inappropriately processing any spuriouslyemitted data is significantly reduced (e.g., potentially fullyeliminated).

During reconfiguration of role 1313, acceleration component 1303 canshut down an application layer for role 1313 and close input/output(I/O) for role 1313. If acceleration component 1303 is also programmedwith roles for other services, acceleration component 1303 is partiallyreconfigured by reconfiguring role 1313. However, other roles atacceleration component 1303 can continue to operate as intended whilerole 1313 is reconfigured. For example, acceleration component 1303 canhave a hardened network interface that permits network communicationduring role (re)configuration. Thus, an image for one role can be loadedonto acceleration component 1033 without overwriting images for otherroles or overwriting the hardened network interface.

Turning to FIG. 13D, acceleration component 1303 can returnreconfiguration complete message 1364 to management service 1322 whenreconfiguration of role 1303 is complete. As depicted, role 1313 isreconfigured as role 1313R. Role 1313 and 1313R can provide the similarfunctionality. In one aspect, role 1313R is identical to role 1313. Thatis, the image for role 1313 is reloaded at acceleration component 1303to correct an error or failure. In another aspect, role 1313R is anupdated version of role 1313.

After configuration is complete (e.g., in response to reconfigurationcomplete message 1364), service manager 1322 can send a message toacceleration component 1303 to activate role 1313R. The message caninstruct role 1313R to accept communication from roles 1312 and 1314.

Method 1400 includes sending an activate command to the one or moreneighbor acceleration components of the acceleration component, theactivate command instructing each of the one or more neighboracceleration components to accept communication from the accelerationcomponent (1405). For example, in response to reconfiguration completemessage 1364, service manger 1322 can send activation commands 1371 and1372 to acceleration components 1302 and 1304 respectively. Activationcommands 1371 and 1372 instruct acceleration components 1302 and 1304 to(re)activate roles 1312 and 1314 respectively and accept communicationfrom role 1313R at acceleration component 1303.

Method 1400 includes receiving an indication from each of the one ormore neighbor acceleration components that the neighbor accelerationcomponent is activated (1406). For example, turning to FIG. 13E,acceleration components 1302 and 1304 can send activated messages 1381and 1382 to service manager 1322 when roles 1312 and 1314 are(re)activated. In response, management service 1322 can send activatecommand 1373 to graph 1333. In response to activate command 1373, graph1333 is (re)activated.

Turning to FIG. 13F, roles 1311, 1312, 1313R, and 1314 can be linked tocompose graph 1333. Graph 1333 graph 1333 can again provide serviceacceleration for the service or portion thereof. Acceleration component1303 is now programmed with specified role 1313R to perform part of theservice acceleration.

Accordingly, in general, aspects can be used to reconfiguring anacceleration component (e.g., an FPGA) communicating with othercomponents (e.g., other acceleration components or host components) on alower level interface by notifying the other components that theacceleration component is going to be reconfigured.

Aspects thus help mitigate (and potentially eliminate) the possibilityof introducing and/or propagating instability into a hardwareacceleration plane and/or software plane during acceleration component(e.g., FPGA). Aspects of address the following problems at least inpart: an acceleration component appearing as a failed (e.g., PCIe)device to the host, raising a destabilizing non-maskable interrupt and afailing or reconfiguring acceleration component corrupting the state ofits neighbors by randomly sending traffic that may appear valid. Aservice manager can disable non-maskable interrupts for an accelerationcomponent during reconfiguration. Thus, acceleration components (e.g.,FPGAs) can remain robust to traffic from neighbors with incorrect orincompatible configurations (e.g. “old” data from FPGAs that have notyet been reconfigured).

To prevent spurious data from corrupting neighboring accelerationcomponent, an acceleration component being reconfigured (or ahigher-level software service, for example, service manager 1322) sendsa “TX Halt” message. The TX halt message indicates that neighboracceleration components are to ignore further traffic until the link isre-established. In addition, messages can be delayed a few clock cyclesso that, in case of an unexpected link failure, it can be detected andthe message can be suppressed.

Similarly, when an acceleration component comes out of reconfiguration,it cannot trust that its neighbors are not emitting spurious data. Tohandle this, each acceleration component can come up with “RX Halt”enabled, automatically throwing away any messages for neighbors. Ahigher-level software service, for example, service manager 1322 cannotify each server to release RX Halt once all acceleration componentsin a graph have been configured.

When a datacenter application hangs for any reason, a higher levelservice in the service hierarchy (such as a machine that aggregatesresults) can notice that a set of servers are unresponsive. The higherlevel service can query each server to find its status. If a server isunresponsive, it is put through a sequence of soft reboot, hard reboot,and then flagged for manual service and possible replacement, until themachine starts working correctly. If the server is operating correctly,it responds to the higher level service with information about thehealth of its local acceleration components (e.g., one or more FPGAs)and associated links.

The higher level service can return a vector with error flags forinter-FPGA (or other acceleration component) connections, DRAM status(bit errors and calibration failures), errors in the accelerationcomponent (e.g., FPGA) application, PLL lock issues, PCIe errors, andthe occurrence of a temperature shutdown. This can call also returns themachine IDs of neighbors of an acceleration component (e.g., an FPGA),to test whether the neighboring acceleration components (e.g., FPGAs) ina graph are accessible and that they are the machines that the higherlevel service expects (in case the cables are miswired or unplugged).

Based on this information, the higher level service may update a failedmachine list (including the failure type) and based on the failurelocation and type, determine where to re-locate various applicationroles on the fabric. It is possible that relocation is unnecessary, suchas when the failure occurred on a spare node, or when simplyreconfiguring an acceleration component (e.g., FPGA) in-place issufficient to resolve the hang. The higher level service can then gothrough its reconfiguration process for every acceleration component(e.g., FPGA) associated with that service to clear out any corruptedstate and mapping out any hardware failure or a recurring failure withan unknown cause.

In some aspects, a system includes a hardware acceleration plane, asoftware plane, and a network infrastructure. The hardware accelerationplane includes a configurable fabric of a plurality of accelerationcomponents. The software plane includes a plurality of host componentsrunning software. The network infrastructure is shared by accelerationcomponents in the hardware acceleration plane and host components in thesoftware plane. The network infrastructure is used by accelerationcomponents to communicate directly with one another. Local links connectacceleration components and host components (e.g., in the same server).

The system also includes one or more computer storage devices havingstored thereon computer-executable instructions representing a servicemanager. The service manager is configured to reconfigure anacceleration component. Reconfiguring the acceleration componentincludes determining that it is appropriate to reconfigure a role at theacceleration component based on a detected condition at one or more of:the acceleration component or the role. The acceleration component isincluded in a group of interoperating acceleration components in thehardware acceleration plane. Roles at each acceleration component in thegroup of interoperating acceleration components are linked together tocompose a graph that provides service acceleration for a service.

Reconfiguring the acceleration component includes isolating theacceleration component from neighbor acceleration components in thegroup of interoperating acceleration components. The accelerationcomponent is isolated by sending a halt command to one or more neighboracceleration components of the acceleration component. The halt commandinstructing the one or more neighbor acceleration components to ignorecommunication from the acceleration component. Reconfiguring theacceleration component includes receiving an indication from each of theone or more neighbor accelerations component that the neighboracceleration component is halted.

Reconfiguring the acceleration component includes reconfiguring the roleat the acceleration component. Reconfiguring the acceleration componentincludes sending an activate command to the one or more neighboracceleration components of the acceleration component. The activatecommand instructs the one or more neighbor acceleration components toaccept communication from the acceleration component. Reconfiguring theacceleration component includes receiving an indication from each of theone or more neighbor acceleration components that the neighboracceleration component is activated.

In another aspect, a method for reconfiguring an acceleration componentamong interconnected acceleration components is performed. It isdetermined that it is appropriate to reconfigure a role at theacceleration component based on a detected condition at one or more of:the acceleration component or the role. The acceleration component isincluded in a group of interoperating acceleration components in ahardware acceleration plane. Roles at each acceleration component in thegroup of interoperating acceleration components are linked together tocompose a graph that provides service acceleration for a service.

The acceleration component is isolated from neighbor accelerationcomponents in the group of interoperating acceleration components. Theacceleration component is isolated by sending a halt command to the oneor more neighbor acceleration components of the acceleration component.The halt command instructs the one or more neighbor accelerationcomponents to ignore communication from the acceleration component. Anindication is received from each of the one or more neighboracceleration components that the neighbor acceleration component ishalted.

The role is reconfigured at the acceleration component. An activatecommand is sent to the one or more neighbor acceleration components ofthe acceleration component. The activate command instructs the one ormore neighbor acceleration components to accept communication from theacceleration component. An indication is received from each of the oneor more neighbor acceleration components indicating that the neighboracceleration component is activated.

In a further aspect, a computer program product for use at a computersystem includes one or more computer storage devices having storedthereon computer-executable instructions that, in response to executionat a processor, cause the computer system to implement a method for amethod for reconfiguring an acceleration component among interconnectedacceleration components.

The computer program product includes computer-executable instructionsthat, in response to execution at a processor, cause the computer systemto determine that it is appropriate to reconfigure a role at theacceleration component based on a detected condition at one or more of:the acceleration component or the role. The acceleration component isincluded in a group of interoperating acceleration components in ahardware acceleration plane. Roles at each acceleration component in thegroup of interoperating acceleration components are linked together tocompose a graph that provides service acceleration for a service.

The computer program product includes computer-executable instructionsthat, in response to execution at a processor, cause the computer systemto isolate the acceleration component from neighbor accelerationcomponents in the group of interoperating acceleration components. Theacceleration component is isolated by sending a halt command to the oneor more neighbor acceleration components of the acceleration component.The halt command instructs the one or more neighbor accelerationcomponents to ignore communication from the acceleration component. Thecomputer program product includes computer-executable instructions that,in response to execution at a processor, cause the computer system toreceive an indication from each of the one or more neighbor accelerationcomponents that the neighbor acceleration component is halted.

The computer program product includes computer-executable instructionsthat, in response to execution at a processor, cause the computer systemto reconfigure the role at the acceleration component. The computerprogram product includes computer-executable instructions that, inresponse to execution at a processor, cause the computer system to sendan activate command to the one or more neighbor acceleration componentsof the acceleration component. The activate command instructs the one ormore neighbor acceleration components to accept communication from theacceleration component. The computer program product includescomputer-executable instructions that, in response to execution at aprocessor, cause the computer system to receive an indication from eachof the one or more neighbor acceleration components that the neighboracceleration component is activated.

The described aspects may be implemented in other specific forms withoutdeparting from its spirit or essential characteristics. The describedaspects are to be considered in all respects only as illustrative andnot restrictive. The scope is, therefore, indicated by the appendedclaims rather than by the foregoing description. All changes which comewithin the meaning and range of equivalency of the claims are to beembraced within their scope.

What is claimed:
 1. A method for reconfiguring an acceleration componentamong interconnected acceleration components, the method comprising aservice manager: determining that it is appropriate to reconfigure arole at the acceleration component based on a detected condition at oneor more of: the acceleration component or the role, the accelerationcomponent included in a group of interoperating acceleration componentsin a hardware acceleration plane, roles at each acceleration componentin the group of interoperating acceleration components linked togetherto compose a graph that provides service acceleration for a service;isolating the acceleration component from neighbor accelerationcomponents in the group of interoperating acceleration components, theacceleration component isolated by sending a halt command to one or moreneighbor acceleration components of the acceleration component, the haltcommand instructing the one or more neighbor acceleration components toignore communication from the acceleration component; receiving anindication from each of the one or more neighbor acceleration componentsthat the neighbor acceleration component is halted; reconfiguring therole at the acceleration component; sending an activate command to theone or more neighbor acceleration components of the accelerationcomponent, the activate command instructing the one or more neighboracceleration components to accept communication from the accelerationcomponent; and receiving an indication from each of the one or moreneighbor acceleration components that the neighbor accelerationcomponent is activated.
 2. The method of claim 1, wherein determiningthat it is appropriate to reconfigure a role at the accelerationcomponent comprises detecting an error at the acceleration component. 3.The method of claim 1, further comprising, prior to isolating theacceleration component from the one or more neighbor accelerationcomponents: sending a command to quiesce the graph; and receiving anindication that the graph is quiesced.
 4. The method of claim 3, furthercomprising, subsequent to receiving an indication from each of the oneor more neighbor acceleration components that the neighbor accelerationcomponent is activated, sending an activate command to the graph toreactivate the graph.
 5. The method of claim 1, wherein reconfiguringthe role at the acceleration component comprises: sending areconfiguration command to the acceleration component to reconfigure therole at the acceleration component; and receiving an indication from theacceleration component that the role has been reconfigured.
 6. Themethod claim 1, wherein reconfiguring the role at the accelerationcomponent comprises loading an image file at the acceleration component.7. The method of claim 1, wherein the acceleration component comprises aField Programmable Gate Array (FPGA).
 8. The method of claim 1, whereinthe service acceleration is for a service selected from among: documentranking, data encryption, data compression, speech translation, computervision, or machine learning.
 9. The method of claim 1, wherein thehardware acceleration plane comprises a configurable fabric of FieldProgrammable Gate Arrays (FPGAs).
 10. A computer program product forimplementing a method for reconfiguring an acceleration component amonginterconnected acceleration components, the computer program productcomprising one or more computer storage devices having stored thereoncomputer-executable instructions that, in response to execution at aprocessor, cause the method to be performed, comprising: determine thatit is appropriate to reconfigure a role at the acceleration componentbased on a detected condition at one or more of: the accelerationcomponent or the role, the acceleration component included in a group ofinteroperating acceleration components in a hardware acceleration plane,roles at each acceleration component in the group of interoperatingacceleration components linked together to compose a graph that providesservice acceleration for a service; isolate the acceleration componentfrom neighbor acceleration components in the group of interoperatingacceleration components, the acceleration component isolated by sendinga halt command to one or more neighbor acceleration components of theacceleration component, the halt command instructing the one or moreneighbor acceleration components to ignore communication from theacceleration component; receive an indication from each of the one ormore neighbor acceleration components that the neighbor accelerationcomponent is halted; reconfigure the role at the acceleration component;send an activate command to the one or more neighbor accelerationcomponents of the acceleration component, the activate commandinstructing the one or more neighbor acceleration components to acceptcommunication from the acceleration component; and receive an indicationfrom each of the one or more neighbor acceleration components that theneighbor acceleration component is activated.
 11. The computer programproduct of claim 10, wherein computer-executable instructions that, inresponse to execution, reconfigure the role at the accelerationcomponent reconfigure the role at the acceleration component comprisecomputer-executable instructions that, when executed, load an image fileat the acceleration component.
 12. The computer program product of claim10, further comprising computer-executable instructions that, inresponse to execution: prior to isolating the acceleration componentfrom the one or more neighbor acceleration components: send a command toquiesce the graph; and receive an indication that the graph is quiesced;and subsequent to receiving an indication from each of the one or moreneighbor acceleration components that the neighbor accelerationcomponent is activated: send an activate command to the graph toreactivate the graph.
 13. The computer program product of claim 10,wherein the acceleration component comprises a Field Programmable GateArray (FPGA).
 14. The computer program product of claim 10, wherein theservice acceleration is for a service selected from among: documentranking, data encryption, data compression, speech translation, computervision, or machine learning.
 15. The computer program product of claim10, wherein the hardware acceleration plane comprises a configurablefabric of Field Programmable Gate Arrays (FPGAs).
 16. A system, thesystem including: a hardware acceleration plane including a configurablefabric of a plurality of acceleration components; a software planeincluding a plurality of host components configured to execute software;a network infrastructure shared by acceleration components in thehardware acceleration plane and host components in the software plane,the network infrastructure used by acceleration components tocommunicate directly with one another; one or more computer storagedevices having stored thereon computer-executable instructionsrepresenting a service manager, the service manager configured to:determine that it is appropriate to reconfigure a role at theacceleration component based on a detected condition at one or more of:the acceleration component and the role, the acceleration componentincluded in a group of interoperating acceleration components in thehardware acceleration plane, roles at each acceleration component in thegroup of interoperating acceleration components linked together tocompose a graph that provides service acceleration for a service;isolate the acceleration component from neighbor acceleration componentsin the group of interoperating acceleration components, the accelerationcomponent isolated by sending a halt command to the one or more neighboracceleration components of the acceleration component, the halt commandinstructing the one or more neighbor acceleration components to ignorecommunication from the acceleration component; receiving an indicationfrom each of the one or more neighbor acceleration components that theneighbor acceleration component is halted; reconfiguring the role at theacceleration component; sending an activate command to the one or moreneighbor acceleration components of the acceleration component, theactivate command instructing the one or more neighbor accelerationcomponents to accept communication from the acceleration component; andreceiving an indication from each of the one or more neighboracceleration components that the neighbor acceleration component isactivated.
 17. The system of claim 16, wherein the accelerationcomponent comprises a Field Programmable Gate Array (FPGA).
 18. Thesystem of claim 16, wherein the service acceleration is for a serviceselected from among: document ranking, data encryption, datacompression, speech translation, computer vision, or machine learning.19. The system of claim 16, wherein the service manager being configuredto determine that it is appropriate to reconfigure a role at theacceleration component comprises the service manager being configured todetect an error at the acceleration component.
 20. The system of claim16, wherein the service manager being configured to reconfigure the roleat the acceleration component comprises the service manager beingconfigured to load an image file at the acceleration component.